Systems and methods for processing test results

ABSTRACT

Systems and methods for processing test results. A method of analyzing test results includes receiving a set of test result files, the set of test result files including a plurality of test results. The method also includes identifying a set of data filters based on one or more of the set of test result files or user input. The method further includes generating filtered results based on the set of data filters and the set of test result files, the filtered results including one or more of a subset of the plurality of test results or reordered test results. The method further includes providing a visual representation of the filtered results.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims priority to U.S. Provisional Application No.62/043,979 filed Aug. 29, 2014, entitled SYSTEMS AND METHODS FORPROCESSING TEST RESULTS. The contents of each of the above-referencedapplication(s) are hereby expressly incorporated by reference herein intheir entireties for all purposes.

BACKGROUND

1. Field

The present disclosure generally relates to systems and methods forprocessing test results.

2. Description of Related Art

Test apparatuses may be devices and/or systems that may be used toperform tests on electronic devices and/or components. The testapparatuses may generate test results and/or test result files thatinclude the test results (e.g., the results of the tests performed onthe electronic devices and/or components).

SUMMARY

In some implementations, the present disclosure relates to a method ofanalyzing test data. The method includes receiving a set of test resultfiles, the set of test result files including a plurality of testresults. The method also includes identifying a set of data filtersbased on one or more of the set of test result files or user input. Themethod further includes generating filtered results based on the set ofdata filters and the set of test result files, the filtered resultsincluding one or more of a subset of the plurality of test results orreordered test results. The method further includes providing a visualrepresentation of the filtered results.

In some embodiments, identifying the set of filters includes providing alist of data filters based on the set of test result files.

In some embodiments, identifying the set of filters further includesreceiving the user input, the user input indicating the set of datafilters.

In some embodiments, the plurality of test results are generated by oneor more test apparatuses, the one or more test apparatus including aplurality of test arms.

In some embodiments, generating the filtered results includes obtaininga test schedule of the plurality of tests, the test schedule includingan order of the plurality of tests and identifying the one or more testapparatuses and the plurality of test arms.

In some embodiments, generating the filtered results further includesidentifying tests performed using a first test arm of the plurality oftest arms based on the test schedule.

In some embodiments, generating the filtered results further includesidentifying tests performed by a first testing apparatus from the one ormore testing apparatuses based on the test schedule.

In some embodiments, the set of data filters includes a first datafilter for identifying a test location.

In some embodiments, the set of data filters includes a first datafilter for identifying a type of a test.

In some embodiments, the set of data filters includes a first datafilter for identifying a type of a failure.

In some embodiments, the set of data filters includes a first datafilter for identifying a type of a component being tested.

In some embodiments, the set of data filters includes a first datafilter for identifying an individual test.

In some embodiments, the visual representation includes one or more oftext, an image, a table, a graph, a chart, or a list.

In some embodiments, providing the visual representation includescommunicating with a spreadsheet application to generate the visualrepresentation.

In some embodiments, the reordered test results includes one or more ofplurality of test results in a different order.

In some implementations, the present disclosure relates to a system foranalyzing test data. The system includes a memory and a processorcommunicatively coupled to the memory. The processor is configured toreceive a set of test result files, the set of test result filesincluding a plurality of test results, identify a set of data filtersbased on one or more of the set of test result files or user input,generate filtered results based on the set of data filters and the setof test result files, the filtered results including one or more of asubset of the plurality of test results or reordered test results, andprovide a visual representation of the filtered results.

In some embodiments, the processor is configured to identify the set offilters by providing a list of data filters based on the set of testresult files.

In some embodiments, the processor is further configured to identify theset of filters by receiving the user input, the user input indicatingthe set of data filters.

In some embodiments, the plurality of test results are generated by oneor more test apparatuses, the one or more test apparatus including aplurality of test arms.

In some embodiments, the processor is configured to generate thefiltered results by obtaining a test schedule of the plurality of tests,the test schedule including an order of the plurality of tests andidentifying the one or more test apparatuses and the plurality of testarms.

In some embodiments, the processor is further configured to generate thefiltered results by identifying tests performed using a first test armof the plurality of test arms based on the test schedule.

In some embodiments, the processor is further configured to generate thefiltered results by identifying tests performed by a first testingapparatus from the one or more testing apparatuses based on the testschedule.

In some embodiments, the set of data filters includes a first datafilter for identifying a test location.

In some embodiments, the set of data filters includes a first datafilter for identifying a type of a test.

In some embodiments, the set of data filters includes a first datafilter for identifying a type of a failure.

In some embodiments, the set of data filters includes a first datafilter for identifying a type of a component being tested.

In some embodiments, the set of data filters includes a first datafilter for identifying an individual test.

In some embodiments, the visual representation includes one or more oftext, an image, a table, a graph, a chart, or a list.

In some embodiments, providing the visual representation includescommunicating with a spreadsheet application to generate the visualrepresentation.

In some embodiments, the reordered test results includes one or more ofplurality of test results in a different order.

In some implementations, the present disclosure relates to anon-transitory computer readable storage medium including instructionsthat, when executed by a processor, cause the processor to performoperations. The operations include receiving a set of test result files,the set of test result files including a plurality of test results. Theoperations also include identifying a set of data filters based on oneor more of the set of test result files or user input. The operationsfurther include generating filtered results based on the set of datafilters and the set of test result files, the filtered results includingone or more of a subset of the plurality of test results or reorderedtest results. The operations also include providing a visualrepresentation of the filtered results.

In some embodiments, identifying the set of filters includes providing alist of data filters based on the set of test result files.

In some embodiments, identifying the set of filters further includesreceiving the user input, the user input indicating the set of datafilters.

In some embodiments, the plurality of test results are generated by oneor more test apparatuses, the one or more test apparatus including aplurality of test arms.

In some embodiments, the filtered results includes obtaining a testschedule of the plurality of tests, the test schedule including an orderof the plurality of tests and identifying the one or more testapparatuses and the plurality of test arms.

In some embodiments, generating the filtered results further includesidentifying tests performed using a first test arm of the plurality oftest arms based on the test schedule.

In some embodiments, generating the filtered results further includesidentifying tests performed by a first testing apparatus from the one ormore testing apparatuses based on the test schedule.

In some embodiments, the set of data filters includes a first datafilter for identifying a test location.

In some embodiments, the set of data filters includes a first datafilter for identifying a type of a test.

In some embodiments, the set of data filters includes a first datafilter for identifying a type of a failure.

In some embodiments, the set of data filters includes a first datafilter for identifying a type of a component being tested.

In some embodiments, the set of data filters includes a first datafilter for identifying an individual test.

In some embodiments, the visual representation includes one or more oftext, an image, a table, a graph, a chart, or a list.

In some embodiments, providing the visual representation includescommunicating with a spreadsheet application to generate the visualrepresentation.

In some embodiments, the reordered test results includes one or more ofplurality of test results in a different order.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example system architecture 100, in accordancewith one embodiment of the present disclosure.

FIG. 2 is a diagram illustrating an example test apparatus 200 accordingto one embodiment of the present disclosure.

FIG. 3 is a block diagram illustrating an example test application 135according to one embodiment of the present disclosure.

FIG. 4 is a diagram illustrating an example GUI 400 according to oneembodiment of the present disclosure.

FIG. 5 is a diagram illustrating an example GUI 500 according to oneembodiment of the present disclosure according to one embodiment of thepresent disclosure.

FIG. 6 is a diagram illustrating an example visual representation 600according to one embodiment of the present disclosure.

FIG. 7 is a diagram illustrating an example visual representation 700according to one embodiment of the present disclosure.

FIG. 8 is a diagram illustrating an example visual representation 800according to one embodiment of the present disclosure.

FIG. 9 is a diagram illustrating an example visual representation 900according to one embodiment of the present disclosure.

FIG. 10 is a diagram illustrating an example process 1000 for viewing,visualizing, processing, filtering, and/or analyzing test resultsaccording to one embodiment of the present disclosure.

FIG. 11 illustrates a diagrammatic representation of a machine in theexample form of a computing device.

DETAILED DESCRIPTION OF SOME EMBODIMENTS

The headings provided herein, if any, are for convenience only and donot necessarily affect the scope or meaning of the claimed invention.

Test apparatuses (e.g., devices and/or systems that may be used toperform tests on electronic devices and/or components) may generate testresults and/or test result files that include the test results. Specialpurpose software (e.g., special purpose applications) may often be usedto analyze and/or process the test result files. The special purposesoftware may be expensive and/or difficult to use without specialtraining. The special purpose software may also not be capable offiltering out different types of data and/or information in the testresult files that may be useful to a user.

Disclosed are non-limiting examples of systems and/or methods related toprocessing and/or analyzing test results. In some embodiments, suchtechniques may be implemented in, for example, a test application on acomputing device (e.g., a desktop computer, a laptop computer, a tabletcomputer, a smartphone, etc.). The test application may use data filtersto identify test results of tests performed at one or more testlocations (e.g., sites), to identify test results of test performed by aparticular test apparatus or test arm, to identify test results of aparticular type of test, to identify test results of tests performed ona type of electronic component/device, to identify test results that mayindicate particular failures or types of failures, and/or to identifyparticular parameters and/or values of the tests and/or test results.Although described in the context of processing and/or analyzing testresults for tests performed on electronic components and/or devices, itwill be understood that one or more features of the present disclosuremay also be utilized in other applications.

FIG. 1 illustrates an example system architecture 100, in accordancewith one embodiment of the present disclosure. The system architecture100 includes test sites 110A through 110X, test apparatuses 120A through120X, network 105, and a computing device 130. Examples of computingdevices may include, but are not limited to, devices such as personalcomputers (PCs), server computers, laptops, mobile phones, smart phones,tablet computers, netbook computers etc. In one embodiment, network 105may include a public network (e.g., the Internet), a private network(e.g., a local area network (LAN) or wide area network (WAN)), a wirednetwork (e.g., Ethernet network), a wireless network (e.g., an 802.11network or a Wi-Fi network), a cellular network (e.g., a Long TermEvolution (LTE) network), routers, hubs, switches, computing devices(e.g., a server computer), and/or a combination thereof.

In one embodiment, test sites 110A through 110X may be geographicallocations where electronic components and/or devices (e.g., poweramplifiers, voltage regulators, radio frequency (RF) components/modules,wireless devices such as tablet computers, smartphones, etc.) may betested. For example, the operation of an electronic component may betested to determine whether the electronic component operates properly.Examples of geographical locations may include, but are not limited to,factories, manufacturing plants, buildings, sites (e.g., groups ofbuildings or structures), cities, counties, states, provinces, regions,and countries, etc. Examples of electronic components and/or devices mayinclude, but are not limited to, amplifiers (e.g., power amplifiers),attenuators, circulators, demodulators, detectors, diodes, directionalcouplers, RF front-end modules, isolators, lighting and displaysolutions, mixers, modulators, optocouplers, optoisolators, phaseshifters, power dividers/combiners, power management devices, RFreceivers, RF switches, voltage regulators, etc.

The test sites 110A through 110X include test apparatuses 120A through120X. A test apparatus may be a device or a combination of devices(e.g., a system) that may be used to test the operation, functionality,tolerances, and/or conformance of electronic components and/or devices.In one embodiment, each test site 110A through 110X may include anynumber of test apparatuses and/or may include one or more differenttypes of test apparatuses. Different types of test apparatuses (e.g.,different models of test apparatuses) may be used to test differenttypes of electronic components and/or devices. For example, site 110Amay include ten test apparatuses to test power amplifiers and three testapparatuses to test RF receivers and site 110X may include five testapparatuses to test phase shifters and three test apparatuses to testattenuators.

In one embodiment, a test apparatus (e.g., test apparatus 120A in site110A) may include a plurality of test arms (as illustrated and discussedin more detail in conjunction with FIG. 2). The test arms may includesensing elements (e.g., sensors) that may be used to perform tests onelectronic devices and/or components (as discussed in more detail inconjunction with FIG. 2). A test apparatus with multiple test arms maybe able to perform tests on multiple electronic devices and/orcomponents simultaneously. This may allow the test apparatus to performtests on electronic devices and/or components more quickly.

As discussed above, the test apparatuses 120A through 120X of the sites110A through 110X may be used to test the operation, functionality,tolerances, and/or conformance of electronic components and/or devices.The test apparatuses 120A through 120X may perform (e.g., run, conduct,etc.) the tests and may generate test result files 125. The test resultfiles 125 may include the results of the tests performed by the testapparatuses 120A through 120X of the sites 110A through 110X and/or mayinclude data indicative of the results of the tests. In one embodiment,the test result files 125 may be provided to the computing device 130via the network 105. For example, the test apparatuses 120A through 120Xmay periodically transmit the test results files 125 to the computingdevice 130 (e.g., transmit the test result files 125 when a testapparatus finishes performing a set of tests or transmit the test resultfiles 125 hourly, daily, weekly, etc.). In another example, thecomputing device 130 may request (e.g., send a request) for the testresult files 125 and the test apparatuses 120A through 120X may transmitthe test result files 125 to the computing device 130 based on therequest. In one embodiment, the test result files 125 may be commaseparated value (CSV) files. In another embodiment, the test resultfiles 125 may use other characters (e.g., a “;”, a “*”, etc.) to delimitand/or separate data and/or values in the test result files 125. Inother embodiments, the test result files 125 may be in any number ofdifferent formats.

In embodiment, the test result files 125 may be stored on a data store(not show in the figures). The data store may be may be a memory (e.g.,random access memory), a cache, a drive (e.g., a hard drive), a flashdrive, a database system, and/or another type of component or devicecapable of storing data. The data store may also include multiplestorage components (e.g., multiple drives or multiple databases) thatmay also span multiple computing devices (e.g., multiple servercomputers).

The computing device 130 includes a test application 135. In oneembodiment, the test application 135 may be an application (e.g., anapp, a program, software, a service, etc.) that allows a user of thecomputing device 130 to view, visualize, process, filter, and/or analyzethe test result files 125 more quickly, easily, and/or efficiently. Thetest application 135 may be a cost effective tool for processing,filtering, and/or analyzing the test result files 125. The testapplication 135 may also provide intuitive user interfaces to allowusers to more easily process, filter, and/or analyze the test resultfiles 125. In one embodiment, the test application 135 may operateand/or execute in conjunction with a spreadsheet application. Forexample, the test application 135 may be an add-in module/component forthe spreadsheet application (e.g., plug-in, extension, add-on, etc.).The test application 135 may communicate and/or interact with thespreadsheet application to view, visualize, process, filter, and/oranalyze the test result files 125. In another embodiment, the testapplication 135 may be standalone application (e.g., may be anapplication capable of executing and/or operating by itself).

In one embodiment, the testing application 135 may identify a set ofdata filters (e.g., one or more data filters) that may be used tofilter, analyze, and/or process the test results in the test resultfiles 125. A data filter may allow the test application 135 to identifycertain types of information and/or data that are stored in the testresult files 125. For example, a data filter may allow the testapplication 135 to identify test results that indicate failures or thatindicate particular types of failures, to identify test results for testperformed on a particular type of electronic component and/or device, toidentify test results for tests performed at a particular site, and/orto identify test results for tests performed by a particular testapparatus and/or a particular test arm. In another example, the datafilters may allow the test application 135 to identify particularparameters and/or values of the tests and/or test results. For example,the data filters may allow the test application 135 to identify testsand/or test results where a particular input value (e.g., a particularvoltage) was used. In one embodiment, the data filters may use regularexpressions (e.g., pattern matching) to identify certain types ofinformation and/or data that are stored in the test result files 125.

In another embodiment, the test application 135 may generate filteredresults using the set of data filters and may provide a visualrepresentation of the filtered results to the user. For example, thetest application 135 may provide (e.g., present and/or display) one ormore of text, an image, a picture, a table, a graph, a chart, and a listto the user of the computing device. The filtered results may include asubset of the test results in the test result files 125. For example,the filtered results may include results that indicate a failure or mayinclude results of tests performed at a particular site. The filteredresults may also include reordered test results from the test resultfiles 125. For example, the filtered results may include the testresults (or a subset of the test results) in a different order. Thefiltered results may also include a subset of the data in the testresults files 125. For example, the filtered results may include thevalue of a particular parameter or measurement.

In one embodiment, the test application 135 may use one or more datafilters to identify results performed by a particular set (e.g., one ormore) of test apparatuses and/or a particular set of test arms. Forexample, the test application 135 may obtain a test schedule (or dataindicative of the test schedule). The test schedule may include data toindicate and/or identify which tests were performed by the testapparatuses 120A through 120X, which test arms of the test apparatuses120A through 120X performed the tests, and the order of the tests (e.g.,the order in which the tests were performed by the test apparatuses 120Athrough 120X and the test arms). The test application 135 may use one ormore data filters and the test schedule to identify test results fortests performed at a particular site, tests performed by a particulartest apparatus (e.g., test apparatus 120A), and/or tests performed by aparticular test arm of a test apparatus.

In one embodiment, the test application 135 may use one or more datafilters to identify test results of tests performed at one or more testlocations (e.g., a geographical location, a test site such as test site110A, etc.). In another embodiment, the test application 135 may use oneor more data filters to identify test results of a particular type oftest (e.g., a test to measure the amount of RF produced by an electroniccomponent and/or device). In a further embodiment, the test application135 may use one or more data filters to identify test results of testsperformed on a type of electronic component and/or device (e.g., testsperformed on power amplifiers or tests performed on modulators). In oneembodiment, the test application 135 may use one or more data filters toidentify test results that may indicate particular failures and/or typesof failures. For example, the one or more data filters may identifyfailures where the test result was above a threshold value or below athreshold value. In another embodiment, the test application 135 may useone or more data filters to identify test results for particular teststhat were performed by the test apparatuses 120A through 120X.

In one embodiment, the test application 135 may allow a user of thecomputing device 130 to process, filter, and/or analyze the test resultfiles 125 more quickly, easily, and/or efficiently. The test application135 may use one or more data filters to format, reorder, reorganize,and/or filter the raw data or information in the test files 124 foreasier analysis. The test application 135 may provide information suchas the overall yield of tests performed by the test apparatuses 120Athrough 120X (e.g., the percentage of electronic components and/ordevices that passed and/or failed the tests). The test application 135may also provide information such as the total number of electroniccomponents and/or devices that passed/failed an individual test and thepercentages of passes/failures for an individual test. The testapplication 135 may further provide statistical data such as the mean,median, and standard deviation for the tests performed by the testapparatuses 120A through 120X. The test application 135 may provide aPareto chart (e.g., a chart that includes bars and a line graph) topresent the results (or a subset of the results) of the tests performedby the test apparatuses 120A through 120X. The Pareto chart may includeinformation such as the tests that have the most failure test results(e.g., the top ten failed tests).

As discussed above, the data filters may be used to identify tests thatwere performed at a particular site (e.g., site 110A), performed by aparticular test apparatus (e.g., test apparatus 120A), and/or performedby a test arm of a test apparatus. The data filters may allow the testapplication 135 to provide information such as the yield from aparticular site, the yield from a particular test apparatus, and/or theyield from a particular test arm. This may allow the user identifyproblems with one or more of the test apparatuses 120A through 120X. Forexample, the test application may provide a visual representation (e.g.,a graph) of test results and the graph may indicate that a larger numberof tests performed by a test arm of a test apparatus result in failures.This may indicate that the test arm is not operating properly (e.g., thetest arm is malfunctioning or is not calibrated properly). The visualrepresentation may allow the user to identify the test arm that is notoperating properly. This may also allow the user to identify performancevariations and/or manufacturing process variations for electroniccomponents and/or devices that are tested at a particular site. Forexample, the test application 135 may allow the user to determine thatpower amplifiers tested at site 110A do not perform as well as poweramplifiers manufactured at site 110X. This may indicate that there maybe a problem with the manufacturing process at site 110A.

FIG. 2 is a diagram illustrating an example test apparatus 200 accordingto one embodiment of the present disclosure. As discussed above, thetest apparatus 200 may be a device or a combination of devices (e.g., asystem) that may be used to test the operation, functionality,tolerances, and/or conformance of electronic components and/or devices(e.g., power amplifiers, attenuators, RF receives, RF switches, RFfront-end modules, modulators, demodulators, attenuators, etc.). Anelectronic component and/or device that may be tested by the testapparatus 200 may also be referred to as a device under test (DUT). Thetest apparatus 200 may be located in geographical location (e.g., a testsite, a factory, a manufacturing plant, a building, a city, a county, astate, a province, a region, a country, etc.).

The test apparatus 200 includes three test arms, test arm 210, test arm220, and test arm 230. Each of the test arms 210, 220, and 230 may beused to perform one or more tests on an electronic component and/ordevice. The multiple test arms 210, 220, and 230 may allow the testapparatus 200 simultaneously test the operation, functionality,tolerances, and/or conformance of three electronic components and/ordevices. In one embodiment, the test arms 210, 220, and 230 may bemovable to different X, Y, and/or Z positions/locations. The term “Z”may define a vertical direction and/or distance relative to the DUT. Theterms “X” and “Y” may define lateral directions and/or distancesrelative to the DUT. In some implementations, the value of Z (heightabove the DUT) may be held substantially uniform (e.g., at approximately1 mm) while tests are performed and/or measurements are made atdifferent X and Y locations. The test arms 210, 220, and 230 may beseparately movable. For example, each of the test arms 210, 220, and 230may be movable independent of each other. The test arms 210, 220, and230 may also be moveable in tandem. For example, the test arms 210, 220,and 230 may all move in the same direction at the same time.

Test arm 210 includes a sensing element 211. Test arm 220 includes asensing element 221. Test arm 230 includes a sensing element 231. In oneembodiment, the sensing elements 211, 221, and 231 may be sensorsconfigured to measure and/or test the operation, functionality,tolerances, and/or conformance of electronic components and/or devices.For example, sensing element 211 may be a sensor configured to detect RFsignals, RF noise, electromagnetic interference (EMI), etc.). The sensormay measure RF signals or electromagnetic (EM) emissions that mayemanate from a DUT (e.g., an RF module, a power amplifier, etc.) whilethe DUT is in operation. The sensor may also measure RF signals or EMemissions that emanate from outside the RF module 1215 and may enter theRF module 1215.

FIG. 3 is a block diagram illustrating an example test application 135according to one embodiment of the present disclosure. The testapplication 135 includes a filter module 310, a report module 320, and aschedule module 330. More or less components may be included in the testapplication 135 without loss of generality. For example, two of themodules may be combined into a single module, or one of the modules maybe divided into two or more modules. In one embodiment, one or more ofthe modules may reside on different computing devices (e.g., differentserver computers). The test application 135, the filter module 310, thereport module 320, and the schedule module 330 may each be processinglogic that includes hardware (e.g., circuitry, dedicated logic,programmable logic, microcode, etc.), software (e.g., instructions runon a processor to perform hardware simulation), firmware, or acombination thereof. The test application 135 may be a standaloneapplication or may be an add-in module/component for the spreadsheetapplication (e.g., plug-in, extension, add-on, etc.). When the testapplication 135 is an add-in module/component, the test application maycommunicate and/or interact with a spreadsheet application (e.g., maycommunicate data between the test application 135 and the spreadsheetapplication and/or may use functions, procedures, etc., of thespreadsheet application).

As discussed above, one or more test apparatuses (e.g., test apparatuses120A through 120X illustrated in FIGS. 1 and 2) may perform one or moretests to test the operation, functionality, tolerances, and/orconformance of electronic components and/or devices. The one or moretest apparatuses may generate test result files (e.g., test result files125 illustrated in FIG. 1) and test result files may be provided to thetest application 135 (as discussed above).

In one embodiment, the filter module 310 identify a set of data filtersthat may be used by the test application 135 to process, filter, and/oranalyze test result files (e.g., test result files 125 illustrated inFIG. 1). For example, the filter module 310 may analyze the test resultfiles to determine the type of the test apparatus that performed thetests. The filter module 310 may identify a list (e.g., a set or one ormore data filters) that may be used to analyze and/or process testresult files. For example, a first type of test apparatus (e.g., a testapparatus that tests power amplifiers) may generate test result files ina first format and the filter module 310 may identify a list of datafilters that may be used to process, filter, and/or analyze test filesin the first format. The filter module 310 may present the list of datafilters (or data indicative of the list of data filters) to the user(e.g., may display the list of data filters to the user). The filtermodule 310 may receive user input indicating or identifying the set ofdata filters. For example, the user may provider user input indicatethat the data filters that identify an individual test, that identify atest arm of a test apparatus, and that identify failure test resultsshould be used.

In one embodiment, the schedule module 330 may obtain a test schedulefor a plurality of tests performed by one or more test apparatuses. Forexample, the schedule module 330 may obtain a file that includes thetest schedule (e.g., may retrieve the file from a data base or fromanother computing device). In another example, the schedule module 330may provider a user interface (e.g., a graphical user interface (GUI)that may include various text fields, buttons, menus, etc.) to allowusers to input the test schedule. The test schedule may include data toindicate and/or identify which tests were performed by the one or moretest apparatuses, which test arms of the test apparatuses performed thetests, and/or the order of the tests (e.g., the order in which the testswere performed by the one or more test apparatuses and/or the testarms). For example, referring to FIG. 1, the test schedule may indicatethat a plurality of tests was performed on test apparatuses 120A and120B in site 110A, that tests were performed using test arms 121A and121B of the test apparatuses 120A and 120B, and may indicate the orderin which the test apparatuses 120A and 120B and test arms 121A and 121Bperformed the plurality of tests.

The test application 135 may use one or more data filters and the testschedule to identify test results for tests performed at a particularsite, to identify tests performed by a particular test apparatus (e.g.,test apparatus 120A), and/or to identify tests performed by a particulartest arm of a test apparatus. Based on the test schedule obtained by theschedule module 330, the filter module 310 may identify data filtersthat may be used to identify test results for tests performed at aparticular site, to identify tests performed by a particular testapparatus, and/or to identify tests performed by a particular test armof a test apparatus. For example, a test apparatus at a site may includethree test arms and a test schedule may indicate that the tests areperformed in groups of three such that the first test arm performs everyfirst test, the second test arm performs every second test, and thethird test arm performs every third test. The filter module 310 mayidentify filters allow the test application 135 to identify testsperformed at the site, to identify tests performed by the testapparatus, and to identify tests performed by each individual test arm.For example, each line in a test result file may be a test result. Thefirst, fourth, seventh, tenth, etc., lines may be test results from thefirst test arm. The second, fifth, eighth, eleventh, etc., lines may betest results from the second test arm. The third, sixth, ninth, twelfth,etc., lines may be test results from the third test arm. The one or moredata filters may be used to identify the test results for the first arm(e.g., the first, fourth, seventh, tenth, etc., lines).

In other embodiments, the one or more data filters identified by thefilter module 310 may also be used by the test application 135 toidentify test results of tests performed at one or more test locations,to identify test results of a particular type of test, to identify testresults of tests performed on a type of electronic component/device, toidentify test results that may indicate particular failures, and/or toidentify particular parameters and/or values of the tests and/or testresults.

In one embodiment, the report module 320 may use the one or more datafilters identify by the filter module 310 to process, filter, and/oranalyze the test results and/or test result files. The report module 320may generate filtered test results. The filtered test results mayinclude a subset of the test result and/or test results in a differentorder than the order in the test result files (e.g., reordered testresults). For example, the report module 320 may use the one or moredata filters to generate filtered test results that include test resultsof tests performed on a particular type of electronic device/componentor that include test results of tests performed by a particular testarm. The report module 320 may generate and/or provide a visualrepresentation of the filtered test results. For example, the reportmodule may generate one or more of an image, a picture, text, a table, agraph, a chart, and a list (as illustrated in FIGS. 6-9).

As discussed above, in one embodiment, the test application 135 may bemay be an add-in module/component for a spreadsheet application (e.g.,plug-in, extension, add-on, etc.). The report module 320 maycommunicate/interact with the spreadsheet application and may usefunctions, operations, methods, etc., of the spreadsheet application togenerate the visual representation. For example, the report module 320may use libraries (e.g., dynamically linked libraries (DLLs)),application programming interfaces (APIs), etc., to generate, provide,and/or present the visual representation (e.g., graph, chart, text,etc.) to the user via a GUI of the spreadsheet application.

FIGS. 4-9 illustrate example GUIs and/or visual representations of testresults (or filtered test results) according to embodiments of thepresent disclosure. The example GUIs and visual representations mayinclude text, images, pictures, tables, graphs, charts, lists, textfields, checkboxes, text boxes, buttons, menus, and/or various otheruser interface elements. The example GUIs and/or visual representationsmay be provided, presented, and/or displayed by a test application(e.g., test application 135 illustrated in FIGS. 1 and 3).

FIG. 4 is a diagram illustrating an example GUI 400 according to oneembodiment of the present disclosure. The GUI 400 may be presented,provided, and/or displayed by a test application (as illustrated inFIGS. 1 and 3) and/or by a spreadsheet application. The GUI 400 includesvarious menus (e.g., a “File” dropdown menu, a “Tools” dropdown menu,etc.). The test application may be accessed by using the “QuickView”drop down menu. For example, the user may select and/or click the“QuickView” drop down menu and may click and/or select the menu item 410to use the test application (e.g., to initiate execution of the testapplication or to start the test application). The GUI 400 also includestest results from one or more test files that are received from and/orgenerated by one or more test apparatuses.

FIG. 5 is a diagram illustrating an example GUI 500 according to oneembodiment of the present disclosure according to one embodiment of thepresent disclosure. The GUI 500 may be presented, provided, and/ordisplayed by a test application (e.g., test application 135 illustratedin FIGS. 1 and 3). The GUI 500 includes various menus (e.g., a “File”dropdown menu) and buttons 505, 510, and 515. Button 505 may allow auser to plot a graph and/or a chart for one or more selected tests(e.g., as illustrated in FIGS. 6-9). Button 510 may allow a user to plota running yield graph for electronic devices and/or components that weretested by one or more test apparatuses. Button 515 may allow a user toplot a running yield graph for electronic devices and/or components thatwere tested by one or more test apparatuses in a particular site (asillustrated in FIG. 9). The portion 525 of the GUI 500 may include testresults of test performed by one or more test apparatuses and/or testarms. The test results include information and/or data such the sitewhere the tests were performed, the mean, maximum (e.g., max), theminimum (e.g., min), etc. of the tests results.

FIG. 6 is a diagram illustrating an example visual representation 600according to one embodiment of the present disclosure. The visualrepresentation 600 may be presented, provided, and/or displayed by atest application (e.g., test application 135 illustrated in FIGS. 1 and3). The visual representation 600 may be a graph that shows the testresults for electronic devices and/or components tested at differentsites (e.g., at different geographical locations, different plants,different buildings, etc.). Portion 610 of the visual representation 600indicates the test results of tests performed at a first site. Eachsolid circle indicates a test result (e.g., a measurement in decibels(dB)) of an electronic device and/or component tested at the first site.Portion 620 of the visual representation 600 indicates the test resultsof tests performed at a second site. Each square indicates a test result(e.g., a measurement in decibels (dB)) of an electronic device and/orcomponent tested at the second site. The visual representation 600 alsoincludes two solid bold lines (e.g., two solid and/or thicker lines)that may indicate a passing range. For example, test results that fallbetween the two solid bold lines may be passing test results (e.g.,results of tests that passed). By using the visual representation 600, auser may be able to determine that there may be a problem with one ormore of the test apparatuses, test arms, and/or manufacturing process atthe second site because the second site has a larger number of failedtests.

FIG. 7 is a diagram illustrating an example visual representation 700according to one embodiment of the present disclosure. The visualrepresentation 700 may be presented, provided, and/or displayed by atest application (e.g., test application 135 illustrated in FIGS. 1 and3). The visual representation 700 may be a graph that shows the testresults for electronic devices and/or components tested by differenttest arms of a test apparatus. Portion 710 of the visual representation700 indicates the test results for tests performed by a first test armof the test apparatus. Each solid circle in the portion 710 indicates atest result (e.g., a measurement in decibels (dB)) of an electronicdevice and/or component tested by the first test arm. Portion 720 of thevisual representation 700 indicates the test results for tests performedby a second test arm of the test apparatus. Each square in the portion720 indicates a test result of an electronic device and/or componenttested by the second test arm. Portion 730 of the visual representation700 indicates test results for tests performed by a third test arm ofthe test apparatus. Each solid circle in the portion 730 indicates atest result of an electronic device and/or component tested by the thirdtest arm. Portion 740 of the visual representation 700 indicates thetest results for tests performed by a fourth test arm of the testapparatus. Each square in the portion 740 indicates a test result of anelectronic device and/or component tested by the fourth test arm. Thevisual representation 700 also includes a solid bold line (a solidand/or thicker line) that may indicate a passing threshold. For example,test results are below the solid bold line may be passing test results(e.g., results of tests that passed).

FIG. 8 is a diagram illustrating an example visual representation 800according to one embodiment of the present disclosure. The visualrepresentation 800 may be presented, provided, and/or displayed by atest application (e.g., test application 135 illustrated in FIGS. 1 and3). The visual representation includes table 805 and Pareto graph 810.The table 805 includes information about that number of failed tests andthe percentage of failed tests for different individual tests (e.g., fortest “PAE_MAX_(—)1710,” for test “PAE_MAX_(—)915,” etc.). The Paretograph 810 includes a line graph and a bar chart that provides a visualrepresentation of the data and/or information in the table 805.

FIG. 9 is a diagram illustrating an example visual representation 900according to one embodiment of the present disclosure. The visualrepresentation 900 may be presented, provided, and/or displayed by atest application (e.g., test application 135 illustrated in FIGS. 1 and3). The visual representation 900 is a graph that displays running yieldof different sites (e.g., different geographical locations). The thickersolid line in the visual representation 900 indicates the running yieldfor a first site. The thinner solid line in the visual representationindicates the running yield for a second site.

FIG. 10 is a diagram illustrating an example process 1000 for viewing,visualizing, processing, filtering, and/or analyzing test resultsaccording to one embodiment of the present disclosure. The process 1000may be performed by processing logic that comprises hardware (e.g.,circuitry, dedicated logic, programmable logic, microcode, etc.),software (e.g., instructions run on a processor to perform hardwaresimulation), or a combination thereof. In one embodiment, process 1000may be performed by a test application, as illustrated in FIGS. 1 and 3.In addition, the process 1000 could alternatively be represented as aseries of interrelated states via a state diagram or events.

The process 1000 begins at block 1010 where the test applicationreceives a set of test result files. For example, the test applicationmay obtain the test result files from a database or from one or moretest apparatuses. The test application may identify a set of datafilters based on the set of test files at block 1020. In one embodiment,the test application may identify the set of test filters by providing alist of data filters to a user, as indicated by block 1021. The testapplication may receive user input indicating the set of data filters atblock 1022. For example, the test application may receive user inputindicating a subset of the list of data filters.

At block 1030, the test application may generate filtered results. Forexample, the test application may use the set of data filters toprocess, filter, and/or analyze the test results in the test resultfiles. As discussed above, the data filters may allow the testapplication to identify test results of tests performed at one or moretest locations, to identify test results of a particular type of test,to identify test results of tests performed on a type of electroniccomponent/device, to identify test results that may indicate particularfailures, and/or to identify particular parameters and/or values of thetests and/or test results. In one embodiment, generating the filteredresults may include obtaining a test schedule, as indicated by block1031. For example, the test application may receive a test schedule viauser input and/or by accessing data (e.g., a file including the testschedule) on a database or other computing device. At block 1032, thetest application may use the data filters and the test schedule toidentify tests performed by a particular test apparatus and/or test arm.The test application may provide (e.g., present and/or display) a visualrepresentation (e.g., images, text, charts, graphs, etc.) of thefiltered test results to a user at block 1040. For example, the testapplication may display a GUI that includes the visual representation(as discussed above).

FIG. 11 illustrates a diagrammatic representation of a machine in theexample form of a computing device 1100 within which a set ofinstructions, for causing the machine to perform any one or more of themethodologies discussed herein, may be executed. The computing device1100 may be a mobile phone, a smart phone, a netbook computer, arackmount server, a router computer, a server computer, a personalcomputer, a mainframe computer, a laptop computer, a tablet computer, adesktop computer etc., within which a set of instructions, for causingthe machine to perform any one or more of the methodologies discussedherein, may be executed. In alternative embodiments, the machine may beconnected (e.g., networked) to other machines in a LAN, an intranet, anextranet, or the Internet. The machine may operate in the capacity of aserver machine in client-server network environment. The machine may bea personal computer (PC), a set-top box (STB), a server, a networkrouter, switch or bridge, or any machine capable of executing a set ofinstructions (sequential or otherwise) that specify actions to be takenby that machine. Further, while only a single machine is illustrated,the term “machine” shall also be taken to include any collection ofmachines that individually or jointly execute a set (or multiple sets)of instructions to perform any one or more of the methodologiesdiscussed herein.

The example computing device 1100 includes a processing device (e.g., aprocessor, a central processing unit (CPU), etc.) 1102, a main memory1104 (e.g., read-only memory (ROM), flash memory, dynamic random accessmemory (DRAM) such as synchronous DRAM (SDRAM)), a network interfacedevice 1108, an output device, 1110, an input device 1112, and a datastorage device 1118, which communicate with each other via a bus 1130.

Processing device 1102 represents one or more general-purpose processingdevices such as a microprocessor, central processing unit, or the like.More particularly, the processing device 1102 may be a complexinstruction set computing (CISC) microprocessor, reduced instruction setcomputing (RISC) microprocessor, very long instruction word (VLIW)microprocessor, or a processor implementing other instruction sets orprocessors implementing a combination of instruction sets. Theprocessing device 1102 may also be one or more special-purposeprocessing devices such as an application specific integrated circuit(ASIC), a field programmable gate array (FPGA), a digital signalprocessor (DSP), network processor, or the like. The processing device1102 is configured to execute test application 1126 for performing theoperations and steps discussed herein.

The computing device 1100 may further include a network interface device1108 which may communicate with a network (e.g., network 105 illustratedin FIG. 1). The computing device 1100 also may include an output device1110 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)),and an input device 1112 (e.g., a mouse, a keyboard, etc.). In oneembodiment, the output device 1110 and the input device 1112 may becombined into a single component or device (e.g., an LCD touch screen).

The data storage device 1118 may include a computer-readable storagemedium 1128 on which is stored one or more sets of instructions (e.g.,test application 1126) embodying any one or more of the methodologies orfunctions described herein. The test application 1126 may also reside,completely or at least partially, within the main memory 1104 and/orwithin the processing device 1102 during execution thereof by thecomputing device 1100. The main memory 1104 and the processing device1102 may also constitute computer-readable media. The instructions mayfurther be transmitted or received over the via the network interfacedevice 1108.

While the computer-readable storage medium 1128 is shown in an exampleembodiment to be a single medium, the term “computer-readable storagemedium” should be taken to include a single medium or multiple media(e.g., a centralized or distributed database and/or associated cachesand servers) that store the one or more sets of instructions. The term“computer-readable storage medium” shall also be taken to include anymedium that is capable of storing, encoding or carrying a set ofinstructions for execution by the machine and that cause the machine toperform any one or more of the methodologies of the present disclosure.The term “computer-readable storage medium” shall accordingly be takento include, but not be limited to, solid-state memories, optical mediaand magnetic media.

The present disclosure describes various features, no single one ofwhich is solely responsible for the benefits described herein. It willbe understood that various features described herein may be combined,modified, or omitted, as would be apparent to one of ordinary skill.Other combinations and sub-combinations than those specificallydescribed herein will be apparent to one of ordinary skill, and areintended to form a part of this disclosure. Various methods aredescribed herein in connection with various flowchart steps and/orphases. It will be understood that in many cases, certain steps and/orphases may be combined together such that multiple steps and/or phasesshown in the flowcharts may be performed as a single step and/or phase.Also, certain steps and/or phases may be broken into additionalsub-components to be performed separately. In some instances, the orderof the steps and/or phases may be rearranged and certain steps and/orphases may be omitted entirely. Also, the methods described herein areto be understood to be open-ended, such that additional steps and/orphases to those shown and described herein may also be performed.

Some portions of the present disclosure are presented in terms ofalgorithms and symbolic representations of operations on data bitswithin a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here and generally,conceived to be a self-consistent sequence of steps leading to a desiredresult. The steps are those requiring physical manipulations of physicalquantities. Usually, though not necessarily, these quantities take theform of electrical or magnetic signals capable of being stored,transferred, combined, compared and otherwise manipulated. It has provenconvenient at times, principally for reasons of common usage, to referto these signals as bits, values, elements, symbols, characters, terms,numbers, or the like.

Some aspects of the systems and methods described herein mayadvantageously be implemented using, for example, computer software,hardware, firmware, or any combination of computer software, hardware,and firmware. Computer software may comprise computer executable codestored in a computer readable medium (e.g., non-transitory computerreadable medium) that, when executed, performs the functions describedherein. In some embodiments, computer-executable code is executed by oneor more general purpose computer processors. A skilled artisan willappreciate, in light of this disclosure, that any feature or functionthat may be implemented using software to be executed on a generalpurpose computer may also be implemented using a different combinationof hardware, software, or firmware. For example, such a module may beimplemented completely in hardware using a combination of integratedcircuits. Alternatively or additionally, such a feature or function maybe implemented completely or partially using specialized computersdesigned to perform the particular functions described herein ratherthan by general purpose computers.

Multiple distributed computing devices may be substituted for any onecomputing device described herein. In such distributed embodiments, thefunctions of the one computing device are distributed (e.g., over anetwork) such that some functions are performed on each of thedistributed computing devices.

Some embodiments may be described with reference to equations,algorithms, and/or flowchart illustrations. These methods may beimplemented using computer program instructions executable on one ormore computers. These methods may also be implemented as computerprogram products either separately, or as a component of an apparatus orsystem. In this regard, each equation, algorithm, block, or step of aflowchart, and combinations thereof, may be implemented by hardware,firmware, and/or software including one or more computer programinstructions embodied in computer-readable program code logic. As willbe appreciated, any such computer program instructions may be loadedonto one or more computers, including without limitation a generalpurpose computer or special purpose computer, or other programmableprocessing apparatus to produce a machine, such that the computerprogram instructions which execute on the computer(s) or otherprogrammable processing device(s) implement the functions specified inthe equations, algorithms, and/or flowcharts. It will also be understoodthat each equation, algorithm, and/or block in flowchart illustrations,and combinations thereof, may be implemented by special purposehardware-based computer systems which perform the specified functions orsteps, or combinations of special purpose hardware and computer-readableprogram code logic means.

Furthermore, computer program instructions, such as embodied incomputer-readable program code logic, may also be stored in a computerreadable memory (e.g., a non-transitory computer readable medium) thatmay direct one or more computers or other programmable processingdevices to function in a particular manner, such that the instructionsstored in the computer-readable memory implement the function(s)specified in the block(s) of the flowchart(s). The computer programinstructions may also be loaded onto one or more computers or otherprogrammable computing devices to cause a series of operational steps tobe performed on the one or more computers or other programmablecomputing devices to produce a computer-implemented process such thatthe instructions which execute on the computer or other programmableprocessing apparatus provide steps for implementing the functionsspecified in the equation(s), algorithm(s), and/or block(s) of theflowchart(s).

Some or all of the methods and tasks described herein may be performedand fully automated by a computer system. The computer system may, insome cases, include multiple distinct computers or computing devices(e.g., physical servers, workstations, storage arrays, etc.) thatcommunicate and interoperate over a network to perform the describedfunctions. Each such computing device typically includes a processor (ormultiple processors) that executes program instructions or modulesstored in a memory or other non-transitory computer-readable storagemedium or device. The various functions disclosed herein may be embodiedin such program instructions, although some or all of the disclosedfunctions may alternatively be implemented in application-specificcircuitry (e.g., ASICs or FPGAs) of the computer system. Where thecomputer system includes multiple computing devices, these devices may,but need not, be co-located. The results of the disclosed methods andtasks may be persistently stored by transforming physical storagedevices, such as solid state memory chips and/or magnetic disks, into adifferent state.

Embodiments of the disclosure also relate to an apparatus for performingthe operations herein. This apparatus may be specially constructed forthe required purposes, or it may comprise a general purpose computerselectively activated or reconfigured by a computer program stored inthe computer. Such a computer program may be stored in a non-transitorycomputer readable storage medium, such as, but not limited to, any typeof disk including floppy disks, optical disks, CD-ROMs andmagnetic-optical disks, read-only memories (ROMs), random accessmemories (RAMs), magnetic or optical cards, flash memory, or any type ofmedia suitable for storing electronic instructions.

The algorithms and displays presented herein are not inherently relatedto any particular computer or other apparatus. Various general purposesystems may be used with programs in accordance with the teachingsherein, or it may prove convenient to construct a more specializedapparatus to perform the required method steps. The required structurefor a variety of these systems will appear from the description below.In addition, the present disclosure is not described with reference toany particular programming language. It will be appreciated that avariety of programming languages may be used to implement the teachingsof the disclosure as described herein.

Unless the context clearly requires otherwise, throughout thedescription and the claims, the words “comprise,” “comprising,” and thelike are to be construed in an inclusive sense, as opposed to anexclusive or exhaustive sense; that is to say, in the sense of“including, but not limited to.” The word “coupled”, as generally usedherein, refers to two or more elements that may be either directlyconnected, or connected by way of one or more intermediate elements.Additionally, the words “herein,” “above,” “below,” and words of similarimport, when used in this application, shall refer to this applicationas a whole and not to any particular portions of this application. Wherethe context permits, words in the above Detailed Description using thesingular or plural number may also include the plural or singular numberrespectively. The word “or” in reference to a list of two or more items,that word covers all of the following interpretations of the word: anyof the items in the list, all of the items in the list, and anycombination of the items in the list. The word “exemplary” is usedexclusively herein to mean “serving as an example, instance, orillustration.” Any implementation described herein as “exemplary” is notnecessarily to be construed as preferred or advantageous over otherimplementations. Furthermore, the terms “first,” “second,” “third,”“fourth,” etc., as used herein are meant as labels to distinguish amongdifferent elements and may not necessarily have an ordinal meaningaccording to their numerical designation.

The disclosure is not intended to be limited to the implementationsshown herein. Various modifications to the implementations described inthis disclosure may be readily apparent to those skilled in the art, andthe generic principles defined herein may be applied to otherimplementations without departing from the spirit or scope of thisdisclosure. The teachings of the invention provided herein may beapplied to other methods and systems, and are not limited to the methodsand systems described above, and elements and acts of the variousembodiments described above may be combined to provide furtherembodiments. Accordingly, the novel methods and systems described hereinmay be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the methods andsystems described herein may be made without departing from the spiritof the disclosure. The accompanying claims and their equivalents areintended to cover such forms or modifications as would fall within thescope and spirit of the disclosure.

1. A method of analyzing test data, the method comprising: receiving aset of test result files, the set of test result files including aplurality of test results; identifying a set of data filters based onone or more of the set of test result files or user input; generatingfiltered results based on the set of data filters and the set of testresult files, the filtered results including one or more of a subset ofthe plurality of test results or reordered test results; and providing avisual representation of the filtered results.
 2. The method of claim 1wherein identifying the set of filters comprises providing a list ofdata filters based on the set of test result files.
 3. The method ofclaim 2 wherein identifying the set of filters further comprisesreceiving the user input, the user input indicating the set of datafilters.
 4. The method of claim 1 wherein the plurality of test resultsare generated by one or more test apparatuses, the one or more testapparatus including a plurality of test arms.
 5. The method of claim 4wherein generating the filtered results comprises obtaining a testschedule of the plurality of tests, the test schedule including an orderof the plurality of tests and identifying the one or more testapparatuses and the plurality of test arms.
 6. The method of claim 5wherein generating the filtered results further comprises identifyingtests performed using a first test arm of the plurality of test armsbased on the test schedule.
 7. The method of claim 5 wherein generatingthe filtered results further comprises identifying tests performed by afirst testing apparatus from the one or more testing apparatuses basedon the test schedule.
 8. The method of claim 1 wherein the set of datafilters comprises a first data filter for identifying a test location.9. The method of claim 1 wherein the set of data filters comprises afirst data filter for identifying a type of a test.
 10. The method ofclaim 1 wherein the set of data filters comprises a first data filterfor identifying a type of a failure.
 11. The method of claim 1 whereinthe set of data filters comprises a first data filter for identifying atype of a component being tested.
 12. The method of claim 1 wherein theset of data filters comprises a first data filter for identifying anindividual test.
 13. The method of claim 1 wherein the visualrepresentation comprises one or more of text, an image, a table, agraph, a chart, or a list.
 14. The method of claim 1 wherein providingthe visual representation comprises communicating with a spreadsheetapplication to generate the visual representation.
 15. The method ofclaim 1, wherein the reordered test results comprises one or more ofplurality of test results in a different order.
 16. A system foranalyzing test data, the system comprising: a memory; and a processorcommunicatively coupled to the memory, the processor configured toreceive a set of test result files, the set of test result filesincluding a plurality of test results, identify a set of data filtersbased on one or more of the set of test result files or user input,generate filtered results based on the set of data filters and the setof test result files, the filtered results including one or more of asubset of the plurality of test results or reordered test results, andprovide a visual representation of the filtered results.
 17. The systemof claim 16 wherein the processor is configured to identify the set offilters by providing a list of data filters based on the set of testresult files.
 18. The system of claim 17 wherein the processor isfurther configured to identify the set of filters by receiving the userinput, the user input indicating the set of data filters.
 19. (canceled)20. The system of claim 19 wherein the processor is configured togenerate the filtered results by obtaining a test schedule of theplurality of tests, the test schedule including an order of theplurality of tests and identifying the one or more test apparatuses andthe plurality of test arms. 21-30. (canceled)
 31. A non-transitorycomputer readable storage medium comprising instructions that, whenexecuted by a processor, cause the processor to perform operationscomprising: receiving a set of test result files, the set of test resultfiles including a plurality of test results; identifying a set of datafilters based on one or more of the set of test result files or userinput; generating filtered results based on the set of data filters andthe set of test result files, the filtered results including one or moreof a subset of the plurality of test results or reordered test results;and providing a visual representation of the filtered results. 32-45.(canceled)